Getting and analyzing remotely sensed open data

2023.05.16, 2023.05.23

Our last tutorial will show you how to navigate through STAC catalogs and access cloud satellite data sets based on your needs.

Goals

1. Use Intake-STAC to browse a STAC catalog and load the data

2. TBD - Make a video showing the time lapse of the Taoyuan international airport?

Installation

Intake and hvplot.

!pip install intake-stac hvplot

Goal 1 procedure

Let’s import the necessary modules:

import intake
import hvplot.xarray

STAC

Planet Lab

Intake

PySTAC and Intake

catalog_url = 'https://www.planet.com/data/stac/catalog.json'

STAC browser

https://radiantearth.github.io/stac-browser

Hurricane Harvey, one of the most devastating natural disasters in the U.S. 2017 Houston

catalog = intake.open_stac_catalog(catalog_url)
catalog

planet:
  args:
    stac_obj: https://www.planet.com/data/stac/catalog.json
  description: ''
  driver: intake_stac.catalog.StacCatalog
  metadata:
    description: This catalog serves as the main root STAC Catalog for Planet Labs.
      It links to 4 small static catalogs of open data, including a small set of Planet
      Disaster Data, some CC-BY SkySat collects and the complete Spacenet 7 images
      and labels.
    id: planet
    stac_extensions:
    - https://stac-extensions.github.io/stats/v0.2.0/schema.json
    stac_version: 1.0.0
    stats:catalogs:
      count: 9
      versions:
        1.0.0: 9
    stats:collections:
      count: 7
      versions:
        1.0.0: 7
    stats:items:
      assets:
        application/geo+json: 1423
        application/json: 26
        image/jpeg: 1
        image/png: 13
        image/tiff: 2
        image/tiff; application=geotiff: 2253
        image/tiff; application=geotiff; profile=cloud-optimized: 425
        text/xml: 3
      count: 3708
      extensions:
        https://stac-extensions.github.io/eo/v1.0.0/schema.json: 396
        https://stac-extensions.github.io/label/v1.0.0/schema.json: 1423
        https://stac-extensions.github.io/projection/v1.0.0/schema.json: 385
        https://stac-extensions.github.io/raster/v1.0.0/schema.json: 3
        https://stac-extensions.github.io/raster/v1.1.0/schema.json: 371
        https://stac-extensions.github.io/view/v1.0.0/schema.json: 31
      versions:
        1.0.0: 3708
    title: Planet Root STAC Catalog
    type: Catalog

list(catalog)

['planet-stac-skysat',
 'planet-disaster-data',
 'sn7',
 'planet/fusion/14N/29E-188N',
 'education']

collection = catalog['planet-disaster-data']
collection

planet-disaster-data:
  args:
    stac_obj: https://www.planet.com/data/stac/disasters/collection.json
  description: '[Planet Disaster Data](https://www.planet.com/disasterdata/) makes
    imagery available directly to the public, volunteers, humanitarian organizations,
    and other coordinating bodies in support of the International Charter for Space
    and Major Disasters. Data is released for individual disaster events, providing
    a 30 day window pre- and post-disaster. Imagery is provided under Creative Commons
    licenses, free of charge, with either CC-BY-SA or CC-BY-NC. This STAC catalog
    provides fully public access of a very small subset of the imagery, as a proof
    of concept.'
  driver: intake_stac.catalog.StacCollection
  metadata:
    catalog_dir: ''

collection.metadata

{'type': 'Collection',
 'id': 'planet-disaster-data',
 'stac_version': '1.0.0',
 'description': '[Planet Disaster Data](https://www.planet.com/disasterdata/) makes imagery available directly to the public, volunteers, humanitarian organizations, and other coordinating bodies in support of the International Charter for Space and Major Disasters. Data is released for individual disaster events, providing a 30 day window pre- and post-disaster. Imagery is provided under Creative Commons licenses, free of charge, with either CC-BY-SA or CC-BY-NC. This STAC catalog provides fully public access of a very small subset of the imagery, as a proof of concept.',
 'stac_extensions': [],
 'title': 'Planet Disaster Data',
 'keywords': ['disaster', 'open'],
 'summaries': {'platform': ['SS02', 'SSC1', '101c'],
  'constellation': ['skysat', 'planetscope'],
  'eo:cloud_cover': {'minimum': 0, 'maximum': 24},
  'eo:gsd': {'minimum': 0.9, 'maximum': 3.7},
  'view:off_nadir': {'minimum': 0.2, 'maximum': 27.3},
  'view:sun_azimuth': {'minimum': 122, 'maximum': 231.9},
  'view:sun_elevation': {'minimum': 56.3, 'maximum': 65.1}},
 'providers': [{'name': 'Planet',
   'description': 'Contact Planet at https://www.planet.com/contact-sales/',
   'roles': ['producer', 'processor'],
   'url': 'https://www.planet.com'},
  {'name': 'Planet Disaster Team <disaster-team@planet.com>',
   'description': 'The Planet Disaster Data Team has released this data as CC-BY-SA-4.0 to help disaster response',
   'roles': ['licensor'],
   'url': 'https://www.planet.com/disasterdata/'},
  {'name': 'Google Cloud Platform',
   'description': 'Hosting is done on a GCP storage bucket owned by the Planet Disaster Data team',
   'roles': ['host'],
   'url': 'https://storage.googleapis.com/pdd-stac/'}],
 'extent': {'spatial': {'bbox': [[-96, 28, -93, 31]]},
  'temporal': {'interval': [['2017-08-28T10:00:00Z', None]]}},
 'license': 'CC-BY-SA-4.0'}

item = collection['hurricane-harvey']['hurricane-harvey-0831']['Houston-East-20170831-103f-100d-0f4f-RGB']
item

mosaic:
  args:
    chunks: {}
    urlpath: https://storage.googleapis.com/pdd-stac/disasters/hurricane-harvey/0831/Houston-East-20170831-103f-100d-0f4f-3-band.tif
  description: 3 Band RGB Mosaic
  direct_access: allow
  driver: intake_xarray.raster.RasterIOSource
  metadata:
    href: https://storage.googleapis.com/pdd-stac/disasters/hurricane-harvey/0831/Houston-East-20170831-103f-100d-0f4f-3-band.tif
    plots:
      geotiff:
        cmap: viridis
        data_aspect: 1
        dynamic: true
        frame_width: 500
        kind: image
        rasterize: true
        x: x
        y: y
    roles:
    - data
    title: 3 Band RGB Mosaic
    type: image/tiff; application=geotiff; profile=cloud-optimized
thumbnail:
  args:
    chunks: {}
    urlpath: https://storage.googleapis.com/pdd-stac/disasters/hurricane-harvey/0831/Houston-East-20170831-103f-100d-0f4f-3-band.png
  description: Thumbnail
  direct_access: allow
  driver: intake_xarray.image.ImageSource
  metadata:
    href: https://storage.googleapis.com/pdd-stac/disasters/hurricane-harvey/0831/Houston-East-20170831-103f-100d-0f4f-3-band.png
    plots:
      thumbnail:
        bands: channel
        data_aspect: 1
        flip_yaxis: true
        kind: rgb
        x: x
        xaxis: false
        y: y
        yaxis: false
    roles:
    - thumbnail
    title: Thumbnail
    type: image/png

thumbnail = item['thumbnail']
# make a note for type(thumbnail)

thumbnail.urlpath

'https://storage.googleapis.com/pdd-stac/disasters/hurricane-harvey/0831/Houston-East-20170831-103f-100d-0f4f-3-band.png'

da_thumbnail = thumbnail.to_dask()
da_thumbnail

<xarray.DataArray (y: 552, x: 549, channel: 3)>
dask.array<xarray-<this-array>, shape=(552, 549, 3), dtype=uint8, chunksize=(552, 549, 3), chunktype=numpy.ndarray>
Coordinates:
  * y        (y) int64 0 1 2 3 4 5 6 7 8 ... 543 544 545 546 547 548 549 550 551
  * x        (x) int64 0 1 2 3 4 5 6 7 8 ... 540 541 542 543 544 545 546 547 548
  * channel  (channel) int64 0 1 2

xarray.DataArray

• y: 552
• x: 549
• channel: 3

• dask.array<chunksize=(552, 549, 3), meta=np.ndarray>

  Array Chunk Bytes 887.84 kiB 887.84 kiB Shape (552, 549, 3) (552, 549, 3) Dask graph 1 chunks in 1 graph layer Data type uint8 numpy.ndarray

• Coordinates: (3)
  • y
    (y)
    int64
    0 1 2 3 4 5 ... 547 548 549 550 551

    array([  0,   1,   2, ..., 549, 550, 551])

  • x
    (x)
    int64
    0 1 2 3 4 5 ... 544 545 546 547 548

    array([  0,   1,   2, ..., 546, 547, 548])

  • channel
    (channel)
    int64
    0 1 2

    array([0, 1, 2])

• Indexes: (3)
  • y
    PandasIndex

    PandasIndex(Int64Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
                ...
                542, 543, 544, 545, 546, 547, 548, 549, 550, 551],
               dtype='int64', name='y', length=552))

  • x
    PandasIndex

    PandasIndex(Int64Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
                ...
                539, 540, 541, 542, 543, 544, 545, 546, 547, 548],
               dtype='int64', name='x', length=549))

  • channel
    PandasIndex

    PandasIndex(Int64Index([0, 1, 2], dtype='int64', name='channel'))

• Attributes: (0)

da_thumbnail.hvplot.image(x='x', y='y', data_aspect=1)

mosaic = item['mosaic']
da_mosaic = mosaic.to_dask()
da_mosaic

<xarray.DataArray (band: 3, y: 22094, x: 21984)>
dask.array<open_rasterio-501c37a1d96e315c20253204d9aec1ce<this-array>, shape=(3, 22094, 21984), dtype=uint8, chunksize=(3, 22094, 21984), chunktype=numpy.ndarray>
Coordinates:
  * band     (band) int64 1 2 3
  * y        (y) float64 3.524e+06 3.524e+06 3.524e+06 ... 3.447e+06 3.447e+06
  * x        (x) float64 -1.066e+07 -1.066e+07 ... -1.058e+07 -1.058e+07
Attributes: (12/13)
    transform:               (3.4638558402435815, 0.0, -10657435.586420376, 0...
    crs:                     +init=epsg:3857
    res:                     (3.4638558402435815, 3.4638558402435815)
    is_tiled:                1
    nodatavals:              (nan, nan, nan)
    scales:                  (1.0, 1.0, 1.0)
    ...                      ...
    AREA_OR_POINT:           Area
    TIFFTAG_DATETIME:        2017:09:01 15:10:49
    TIFFTAG_RESOLUTIONUNIT:  2 (pixels/inch)
    TIFFTAG_SOFTWARE:        Adobe Photoshop CC 2015.5 (Macintosh)
    TIFFTAG_XRESOLUTION:     72
    TIFFTAG_YRESOLUTION:     72

xarray.DataArray

• band: 3
• y: 22094
• x: 21984

• dask.array<chunksize=(3, 22094, 21984), meta=np.ndarray>

  Array Chunk Bytes 1.36 GiB 1.36 GiB Shape (3, 22094, 21984) (3, 22094, 21984) Dask graph 1 chunks in 2 graph layers Data type uint8 numpy.ndarray

• Coordinates: (3)
  • band
    (band)
    int64
    1 2 3

    array([1, 2, 3])

  • y
    (y)
    float64
    3.524e+06 3.524e+06 ... 3.447e+06

    array([3523817.157214, 3523813.693358, 3523810.229502, ..., 3447297.117847,
           3447293.653991, 3447290.190136])

  • x
    (x)
    float64
    -1.066e+07 ... -1.058e+07

    array([-10657433.854492, -10657430.390637, -10657426.926781, ...,
           -10581294.839268, -10581291.375412, -10581287.911556])

• Indexes: (3)
  • band
    PandasIndex

    PandasIndex(Int64Index([1, 2, 3], dtype='int64', name='band'))

  • y
    PandasIndex

    PandasIndex(Float64Index([ 3523817.157214148, 3523813.6933583077, 3523810.2295024674,
                   3523806.765646627,  3523803.301790787, 3523799.8379349466,
                  3523796.3740791064,  3523792.910223266,  3523789.446367426,
                  3523785.9825115856,
                  ...
                  3447321.3648382085, 3447317.9009823683, 3447314.4371265285,
                   3447310.973270688,  3447307.509414848, 3447304.0455590077,
                  3447300.5817031674,  3447297.117847327,  3447293.653991487,
                  3447290.1901356466],
                 dtype='float64', name='y', length=22094))

  • x
    PandasIndex

    PandasIndex(Float64Index([-10657433.854492456, -10657430.390636615, -10657426.926780775,
                  -10657423.462924935, -10657419.999069095, -10657416.535213254,
                  -10657413.071357414, -10657409.607501574, -10657406.143645734,
                  -10657402.679789893,
                  ...
                  -10581319.086258944, -10581315.622403104, -10581312.158547264,
                  -10581308.694691423, -10581305.230835583, -10581301.766979743,
                  -10581298.303123903, -10581294.839268062, -10581291.375412222,
                  -10581287.911556382],
                 dtype='float64', name='x', length=21984))

• Attributes: (13)

  transform :

  (3.4638558402435815, 0.0, -10657435.586420376, 0.0, -3.4638558402435815, 3523818.889142068)

  crs :

  +init=epsg:3857

  res :

  (3.4638558402435815, 3.4638558402435815)

  is_tiled :

  1

  nodatavals :

  (nan, nan, nan)

  scales :

  (1.0, 1.0, 1.0)

  offsets :

  (0.0, 0.0, 0.0)

  AREA_OR_POINT :

  Area

  TIFFTAG_DATETIME :

  2017:09:01 15:10:49

  TIFFTAG_RESOLUTIONUNIT :

  2 (pixels/inch)

  TIFFTAG_SOFTWARE :

  Adobe Photoshop CC 2015.5 (Macintosh)

  TIFFTAG_XRESOLUTION :

  72

  TIFFTAG_YRESOLUTION :

  72

da_mosaic_subset = da_mosaic.sel(x=slice(-10641000, -10638000), y=slice(3474000, 3471000))
da_mosaic_subset.hvplot.rgb(x='x', y='y', data_aspect=1)